How To Determine Whether The Us Cn2 Server Cannot Make Calls In Some Areas Because Of Routing Or Firewall Issues

1.

preparation and tools required

please prepare a host that can initiate testing from the "problem area" (preferably multiple locations: north and south china, china mobile/telecom/china unicom), tools: ping, traceroute (linux), tracert (windows), mtr, tcptraceroute/tcpsing/tcping, nmap, telnet/nc, tcpdump/wireshark, bgp looking glass (such as bgp.he.net, major operators looking glass). if you don't have linux, you can use wsl or a convenient online route detection service.

2.

step one: basic connectivity (ping) verification

on the machine in the problem area, execute: ping -c 5 server_ip or windows: ping -n 5 server_ip. pay attention to recording the packet loss rate and delay. if ping fails completely but works in other areas, it means there is packet loss or unreachability; if only partial packet loss occurs, the route may be unstable or icmp is restricted.

3.

step 2: simple path tracking (traceroute/tracert) comparison

linux: traceroute -i server_ip (icmp), traceroute -t -p 443 server_ip (tcp 443), traceroute -u server_ip (udp); windows: tracert server_ip, or powershell: test-netconnection -traceroute -port 443 -computername server_ip. comparing the hop count and timeout position of icmp and tcp, if icmp arrives but tcp times out at a certain hop, the firewall tends to drop tcp in the middle.

4.

step 3: use mtr for continuous routing and packet loss analysis

execute mtr -rwzbc 100 server_ip (linux) or mtr --tcp -p 443 server_ip (some mtr supports tcp). mtr will show which hop the packet loss started and whether it continues. if the packet loss continues to increase from a certain asn/router, it usually indicates that there is a routing quality or policy problem; if the performance is different on multiple ports, it may be a firewall filtering by port.

5.

step 4: port-level connectivity test (tcp/udp)

use tcping/tcptraceroute/nc: tcping server_ip 443 or nc -vz server_ip 443 (linux). for udp services use nmap -su -p port server_ip. if tcp 443 is connectable but icmp is unreachable, it may be due to the operator or the remote icmp policy; if tcp 443 is unreachable but traceroute shows that the path is normal, it may be that the target or intermediate firewall is filtering specific ports.

6.

step 5: try changing the protocol and port

make three sets of comparisons: icmp, tcp (80/443/others), and udp. use traceroute to specify the protocols separately, or use curl -v --connect-timeout 10 https://server_ip:443. if some tcp/udp ports are allowed to pass but other ports are blocked, it indicates that there is a port-level firewall policy; if all protocols are disconnected at the same hop, it is more likely to be a routing problem.

7.

step 6: check mtu and pmtu issues

packet loss on some paths or failure to establish https may be a pmtu blackhole. use ping -m do -s size server_ip to gradually reduce size (linux), or use ping -f -l size on windows. if large packets cannot pass but small packets can, it indicates an mtu problem. you can enable mss clamping on the server/router (such as setting tcp mss 1360 on the router) or adjust the network card mtu.

8.

step 7: local and server firewall check (local)

linux check: iptables -l -n -v, nft list ruleset, ufw status verbose; windows check: netsh advfirewall firewall show rule name=all or powershell get-netfirewallrule. confirm that there are no mistakenly blocked policies, temporarily turn off the firewall to test connectivity (pay attention to security risks), or use iptables -i input -s client_ip -j accept to do a whitelist test.

9.

step 8: server-side and cloud provider security group check (remote)

log in to the server or cloud console to check the security group/acl (such as aws sg, alibaba cloud security group, tencent cloud security group), confirm that the source ip/port is allowed, check fail2ban, csf, iptables logs (/var/log/messages or /var/log/secure, /var/log/iptables.log), and check whether there are rejection records (reject, drop, rst). if the server is in the united states and uses cn2 for backhaul, confirm whether the cloud provider or server room has policy restrictions on certain domestic asns.

10.

step 9: packet capture analysis (tcpdump/wireshark)

capture packets on the server and client respectively: sudo tcpdump -nni eth0 host client_ip and port 443 -w /tmp/cap.pcap. observe whether the three-way handshake is completed, whether there is icmp unreachable (type3 code4, etc.), whether there are rst packets or a large number of syn but no ack. if the client sends syn to the server but the server does not respond or responds with icmp unreachable, it may be a routing or intermediate firewall problem; if the server receives syn and sends rst, it means that the server firewall or service is not listening.

11.

step 10: bgp and routing view (as routing level)

use looking glass (such as bgp.he.net, each operator lg) to check the as routing paths from different locations to the target to see if there are black holes or inconsistent as paths. use whois/bgp.he.net to see if there are any restrictions or communities for certain prefixes. if multiple sources arrive at different prefix paths but are all disconnected at the same node, it may be a policy issue with that node.

12.

step 11: comprehensive judgment rules

determination idea: if the traceroute/tcp path stops at a certain hop and the hop is a specific operator or asn, and the packet capture client segment is not reached, then it is a routing problem; if the server receives the packet but returns rst or directly rejects it, or the server firewall log shows interception, it is a firewall problem; the port differences are obvious (80 is passable and 443 is not), mostly due to firewall port policies.

13.

step 12: address suggestions and temporary workarounds

suggestions for routing problems: submit routing alarms to the upstream or cdn/computer room, provide traceroute/mtr/capture packets and time points, and request bgp engineering intervention; you can temporarily use cdn, reverse proxy, or change the export operator backhaul. suggestions for firewall issues: adjust the security group/iptables rules, allow the source network segment, whitelist the server or modify the fail2ban policy. you can also temporarily enable port tunneling or vpn bypass.

14.

step 13: verify fixes and regression testing

after repair, repeat the above tests: ping, traceroute (icmp/tcp/udp), mtr, tcping, packet capture, and compare the mtr/traceroute output and packet capture results before and after repair. after confirming that the three-way handshake is normal, the application layer can be connected, and there is no obvious packet loss, inform the upstream/customer and continue to monitor for 24-72 hours.

15.

tips on common misunderstandings

don't just look at a single ping result; icmp connectivity does not mean tcp application connectivity; single-point testing may be affected by local isp policies, and it is recommended to test multiple points (different asns) at the same time. keep evidence (packet capture, mtr output, timestamp) to facilitate communication with operators or computer rooms.

16.

q: if the traceroute starts to time out at a certain hop but the mtr shows that it can still be reached, what is the problem?

q: what does it mean when traceroute times out on a certain hop but mtr can be reached? answer: it may be that the hop device limits or discards icmp/ttl expiry messages or udp/icmp responses, but in fact the data packets can pass to farther nodes. this usually means that the router has implemented policy processing on the detection packet (the router itself does not echo it). it cannot rely solely on a single traceroute to assert that the service is unreachable. it should be judged in combination with mtr continuous sampling and server packet capture.

17.

q: when capturing packets, i see that syn reaches the server but no ack. how to determine whether it is a routing or service problem?

question: how to judge when syn reaches the server but there is no ack? answer: if the server network card can receive syn (tcpdump can see it), it means that the route reaches the server; at this time, check whether the server responds (whether the kernel sends syn+ack or rst); if the server does not respond, check the local firewall, tcp wrapper, service listening port or kernel settings; if the kernel sends rst, the service is not monitored or is rejected by the firewall; the conclusion is biased toward server/firewall issues.

18.

q: the other party is cn2 backhaul, and packets are lost in some areas. how should i submit a work order to the computer room/operator?

q: how to report a fault to the computer room/operator and provide effective information? answer: provide: time period of occurrence, affected source ip (at least 2-3 different areas), target ip, complete traceroute (icmp/tcp port 443), mtr output, packet capture (if available) and ping packet loss rate. indicate whether it is a cn2 backhaul and ask to check the policy and bgp announcement of the asn/router. the more detailed, the better for project positioning.